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DvCPROVEMENTS IN OR RELATING TO IMAGE PROCESSING 



The present invention relates to image processing and, in particular, to the 
enhancement of images to assist in their interpretation. 
5 There are many techniques for the processing of images, particularly digitised 

images, to reduce noise and assist in their interpretation. Such techniques are particularly 
important in the field of medical imaging, where images are typically noisy and difficult 
to interpret since clinically significant signs are mostly subtle. As an example, x-ray 
imaging is used as a basis for many medical techniques and, in particular, 

10 mammography is currently the examination of choice for early detection of breast 
cancer. One of the earliest indicators of breast cancer is the presence of 
microcalcifications, which can often be identified in mammograms as localized bright 
spots. In the accompanying drawings Figure 1 illustrates some mammogram samples 
showing microcalcifications. Figures 1 A and 3 show isolated calcifications, while 

15 Figures 1C and D show microcalci£cation clusters. It would be useful to have image 
enhancement techniques which assist radiologists or clinicians in finding 
microcalcifications in mammogram images. However, it is important that such 
techniques miss as few clinically important microcalcification clusters as possible, and 
also do not signal too many false positives. 

20 Methods of detecting automatically microcalcifications in mammograms have 

been proposed, for instance in WO-A-00/52641 and WO-A-0 1/69533. These techniques 
are based on an adapted version of the image known as the h ini representation in which 
the specific imaging parameters particular to the imaging process are removed. This 
results, in essence, in a normalised image known as the Standard Mammogram Format 

25 (SMF) which can be displayed as an h„ u surface, or with the h mr values convened into 
grey levels, in which case the image resembles a conventional mammogram. The 
techniques for producing the h m , representation will not be repeated here, but they are 
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explained in detail in WO-A-00/5264T which is incorporated herein by reference. 
Figures 2 and 3 of the accompanying drawings illustrate respectively an original 
mammogram and the different h im or SMF representations. In Figure 2 the h in , values are 
shown as a surface whereas in Figure 3 the h tni values are converted into grey levels and 
5 displayed as an SMF akin to a conventional mammogram. 

One technique for enhancing images is known as "diffusion". This is, in essence, 
a smoothing process in which the image is processed by convolving the intensity values 
in the image with a kernel for instance a Gaussian kernel. Although such a smoothing 
process can assist in enhancing images, it can also create problems. In particular, in an 

10 image containing an object shown against a background, smoothing or blurring of the 
object into the background is undesirable. Therefore so called "anisotropic diffusion" 
techniques have been proposed in which the diffusion processing occurs within objects, 
and within the background, but not across the boundaries between the two. Such 
techniques are disclosed, for instance, in "Scale-Space and Edge Detection Using 

1 5 Anisotropic Diffusion" by Perona and Malik (IEEE Transactions on Pattern Analysis 
and Machine Intelligence, volume 12, number 7, July 1990) and "Robust Anisotropic 
Diffusion 11 by Black et al. (IEEE Transactions on Image Processing, volume 7, number 
3, March 1998) which are incorporated herein by reference. In these techniques, though, 
the parameters of the diffusion process which include the number of iterations, the scale 

20 of the kernel and the diffusion coefficient itself, are typically set interactively by the 

user. This would be impractical for medical image processing in which many images of 
different qualities and characteristics are produced, thus requiring the automatic setting 
of parameters for each individual image. 

The first aspect of the present invention provides a method of processing 

25 mammogram images using an anisotropic diffusion process in which the anisotropic 
diffusion process is adaptive in dependence upon the image being processed. The 
process adapts itself in accordance with the characteristics of the image, eg a measure of 
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the contrast in the image. This adaptation is automatic, and thus does not require the user 
to assess the image and set the diffusion process parameters for each different type of 
image. 

The diffusion process may be made adaptive by changing its parameters, e.g. its 
5 diffusion coefficient, and/or, for example, the number of iterations in the process. For 
example, the diffusion coefficient may be dependent upon the contrast in the. image. In 
particular it may be calculated from a statistical analysis or measure of the local contrast 
in the image, e.g. based on an average value and standard deviation of the local contrast 
values. 

1 0 Thus the invention involves taking an original image, possibly processing it to 

enhance it using known techniques, such as in a mammogram to produce the Standard 
Mammogram Format, possibly performing other enhancements such as taking the 
Gaussian derivative, and then applying an anisotropic diffusion process in which at least 
one of the parameters of the diffusion process are calculated from the characteristics of 

15 this particular image. 

It is found that this technique allows the application of anisotropic diffusion 
processing to many different images, e.g. different mammograms, on an automated 
basis. In the case of mammograms it provides enhancement of the visibility in the 
processed image of microcalcifications. 

20 Another aspect of the invention provides a method of segmenting an object in an 

image from the background of the image by using a contrast based segmentation 
method, such sis a so-called fovea! segmentation algorithm, in which the segmentation 
algorithm is made adaptive by being dependent upon the characteristics of the image 
being processed, such as the contrast. 

25 Foveal segmentation is a segmentation based on the local contrast in areas of the 

image. It is based on an analysis of human brightness perception as explained in "A New 
[mage Segmentation Method Based on Human Brightness Perception and Fovea! 
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Adaptation" by Heucke et al (IEEE Signal Processing Letters, volume 7, number 6, June 
2000). In the technique described in that paper, areas of an image are assigned to belong 
to cither an object or the background depending on whether the local contrast is above a 
certain minimum contrast. The minimum contrast is calculated to be the minimum 
contrast perceivable by the human eye. With this aspect of the present invention, 
however, the segmentation technique is developed so that at least one of the parameters 
of the segmentation process is calculated from the image characteristics. This allows the 
automatic segmentation of images of different characterises without the need' for the 
user interactively to set the segmentation parameters. Conveniently the minimum 
contrast value is defined with respect to the contrast in the image, for instance a 
statistical analysis or measure of the local contrast in the image, e.g. based on an average 
value and standard deviation of the local contrast values. 

The two aspects of the invention may be combined together and they are 
particularly useful for processing medical x-ray images, particularly digitised 
15 mammograms. 

The invention also extends to a computer program comprising program code 
means for executing the image processing method on a suitably programmed computer 
system, to a computer readable storage medium carrying the computer program, and to 
an image processing apparatus for executing the image processing method. 

The invention will be further described by way of example with reference to the 
accompanying drawings in which:- 

Figures 1 A to D illustrate mammograms including microcalcifications; 
Figures 2A and B illustrate respectively a mammogram and its Standard 
Mammogram Format; 

Figures 3 A and B illustrate a mammogram and its Standard Mammogram 
Format; 

Figure 4 is a flow diagram illustrating image processing according to one 
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embodinient of the present invention; 1 

Figure 5 A shows an example of a mammogram containing a large calcification 
and several artifacts and Figure 5B illustrates the result of diffusing the image of Figure 
5A; 

5 Figure 6A illustrates an SMF image containing a microcalcification, Figure 6B 

the diffused conversion of image A, Figure 6C a 3-D plot of the SMF image of Figure 
6A, and Figure 6D the surface of the diffused image in Figure 6B; 

Figures 7A through D illustrate the removal of artifact from a mammogram, 
namely successively: Figure7A illustrates the original image, Figure 7B a map of the 
10 shot-noise, Figure- 7C a map of curvilinear structures in the image and Figure TD the 
image after shot-noise and curvilinear structure removal; 

Figure 8 illustrates an original image and diffused versions of the image with 
different parameters; 

Figure 9 illustrates an original mammogram and its surface plot together with 
1 5 diffused versions of the mammogram and surface plot; 

Figures 10A and B illustrate original images with calcifications, Figures 10C and 
D illustrate gradient maps for the images of Figures 10A and B and Figures 10E and F 
illustrate diffused versions of the images of Figures 10A and B; 

Figures 1 1 A through J illustrate original SMF images alongside corresponding 
20 processed images in which the microcalcifications have been detected and marked in 
accordance with an embodiment of this invention. 

Figure 4 is a flow diagram of image processing in accordance with one 
embodiment of the invention. In this embodiment the processing according to the 
25 invention is preceded by processing which has previously been proposed to enhance the 
image. 

The grey-level original image 1 is first blurred using a Wiener filter. This de- 
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aoises the image to an extent by removing radiographic mottle, which is a source of false 
positives in detecting microcalcifications. The Wiener filter is adapted to the 
characteristics of radiographic noise in the original image. This technique is explained in 
Yam, M. Brady, J.M. Highnam, R.P. English, R.: Denoising h tnJ Surfaces: a Physics- 
5 based Approach, in Medical Image Computing and Computer-Assisted Intervention 
1999, Spnnger-Verlag, Berlin Heidelberg New York (1999) 227-234, incorporated 
herein by reference. 

The next step is the generation of the Standard Mammogram Format (SMF) 5 
using the technique described in WO-A-00/52641. 

1 0 This may be further processed by the glare removal technique disclosed in WO- 

A-00/52641 to produce the blurred, no glare, SMF 7. 

A major source of errors in detecting microcalcifications is film-screen shot 
noise, which appears primarily from small pieces of dust on the intensifying screen and 
has visual properties which are similar to those of microcalcifications. However, because 

1 5 shot noise is caused, for example, by dust on the screen rather than by structures within 
the breast, it is characterised by the absence of blur. Therefore such shot noise may be 
detected by the absence of blur, and then removed from the image. This technique is 
described in Highnam, R.P. Brady, J.M. English, R_; Detecting Film-Screen Artifacts in 
Mammography using a Model-Based Approach, in IEEE Transactions in Medical 

20 Imaging, Vol. 18 (1999) 1016-1024 which is herein incorporated by reference. Further, 
curvilinear structures in the breast have similar visual properties to microcalcifications 
when viewed in a noi3y image. It is advantageous, therefore, to use one of the available 
techniques for the removal of curvilinear structures, for example based on phase 
congruency as disclosed in Yates, K. Evans, C J. Brady, J.M.: Improving the Brake's 

25 Mammographic Mass Detection Algorithm Using Phase Coogruency, m Proceedings of 
Digital Image Computing: Techniques and Applications, Melbourne (2002), which is 
herein incorporated by reference. 
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This results in an enhanced SMF 9. Figure 7 illustrates this artifact removal 
process. Figure 7A illustrates the original image and Figure 7B the shot-nojse map 
(white dots are noise). Figure 7C jllustrates the curvilinear structure map, and Figure 7D 
the "clean" image after shot-noise and CLS removal. 
5 Next, in accordance with the invention, the clean SMF 9 is subjected to an 

adaptive anisotropic diffusion process to produce a diffused image 1 1, and then to 
adaptive foveal segmentation to produce a map of microcalcifications 13. These 
processes are rendered adaptive by using a parameter k which is representative of the 
local contrast in the image. This parameter is denved from a gradient map 15 whose 
1 0 calculation will be described below. 

The parametric format of anisotropic diffusion makes it highly dependent upon 
the fine-tuning of its input parameters. There are three parameters to be considered when 
attempting to blur an image using anisotropic diffusion: k - the contrast, t - the time or 
number of iterations and a - the standard deviation or scale. In practice, the more 
1 5 complex and variable the image is in a data set, the more problematic it is to choose a 
single set of values for these parameters that work well for the entire data set. Medical 
images, and certainly mammograms, are very complex images whose appearance varies 
widely across a population (at a centre, hospital, region, country or continent), which 
makes the vital requirements of generating few false positives and fewer false negatives 
20 very difficult. 

In accordance with this embodiment of the present invention the contrast 
parameter k, which is image dependent, is varied in dependency upon the characteristics 
of the image. The time parameter t is set to be constant (i.e. a coostant number of 
iterations), as is the scale. 
25 *n accordance with this embodiment of the invention the adaptive anisotropic 

diffusion is conducted with parameters, in particular a contrast value, denved from use 
of a Gaussian derivative filter. Firstly, the SMF 9 is processed to derive the Gaussian 
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derivative of the image in accordance with equations 2 and 3 below:- 




M=\K a \I)\ (3) 



5 where K is the Gaussian of image / and M the Gaussian derivative. ' 

Then the values of the local contrast g t are calculated for the image in accordance 
with equation 4 below:- 



The local contrast is calculated in a neighbourhood of N pixels. These values g ; may be 
displayed in a gradient map as shown in Figure 10 in which Figures 10A and 3 are 
images including respectively an isolated calcification and a microcalcification cluster, 
and Figures 10C and D are the corresponding gradient maps. It can be seen that the 

1 5 calcifications are more visible in the gradient maps. 

In this embodiment a computed contrast value k is then calculated from the 
gradient map for the image in accordance with equation 5 below. This value is set to be 
the average value of the local contrasts plus two standard deviations. This value will be 
subsequently used in the anisotropic diffusion process and also in the foveal 

20 segmentation process. 
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Having calculated the value k to anisotropic diffusion process is applied to the 
clean SMF 9. This involves applying a diffusion tensor similar to that disclosed in 
Weiclcert, J.: Anisotropic Diffusion uvlmage Processing. B.B. Teubner, Stuttgart (1998) 
herein incorporated by reference, but using the eigenvalues below: - 



X, = \for\gradI a \= 0 



f 



\ = 1 - exp 



-i 



[{\gradl a \/k) n ) 



for\gradlJ[> 0 



where / is the initial image, l 9 the Gaussian smoothed image, k the calculated contrast 

measure and n is a suitably high power such as S or 12. It can be seen that where k is 

high, i.e. where the contrast is high, thus indicative of an edge, the value of the 
10 exponential term in X, is very small, thus inhibiting diffusion across the edge. 

Figure 8 illustrates an example of an original image and diffused versions of it 

using k = 5, o = 0.6 and firstly t = 20 iterations and secondly t = 40 iterations. It can be 

seen that the microcalcification and also noise are more visible in the diffused images. 

Figure 9A and B illustrate an original mammogram in Figure 9A and its surface plot in 
1 5 Figure 9B, together with a diffused SMF of the mammogram in Figure 9C and its 

corresponding surface plot in Figure 9D. In thus case the diffusion was conducted with k 

= 15, a = 0.6 and t =5 iterations. 

Figures 5 and 6 also illustrate image diffusion. Figure 5A illustrates a sample of a 

mammogram containing a large calcification and several artifacts. Figure 5B shows the 
20 result of diffusing the image and the smooth background and calcification can be clearly 

distinguished. In Figure 6, Figure 6A is an SMF image containing a microcalcification 
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on the left side and a large spot of noilse on the lower right side, Figure 6B is a diffused 
version of Figure 6 A and Figure 6C is the 3-D plot of the SMF image in Figure 6 A. The 
surface plot shows an extremely noisy appearance and important structures can barely be 
distinguished- However, in Figure 6D|the surface of the diffused image of Figure 6B is 
5 shown and the microcalcification appears as a hill with smoother edges than those of the 
very sharp-edged noise structures in the same image, while the background is smooth. 
Therefore the important structures can be distinguished more easily. 

The final step in the process illustrated in Figure 4 is the application of an 
adaptive segmentation method, such as the foveal method explained by Heucke et ai. 

10 This processing is conducted upon the diffused SMF image 1 1 . A set of mean values of 
the image intensities/values is computed using masks for the inner area, its 
neighbourhood and background. The histogram of the inner surface provides the mean of 
the values in the object (p 0 ), and the histogram of the whole image gives the mean of the 
background values (ji a )- ^ c mean of the values in the neighbourhood (^ N ) is defined as 

1 5 the weighted sum of intensities with a suitable scale set for the mask. Then the 
perceivable contrast is:- 



Mm 



Then a minimum contrast value is computed where j.t A = w * j.i N +(l-w) p. B where w is a 
20 suitable weight between 0 and 1 . In practice w=0.923 gives good results. Thus is 
calculated in accordance with equation 7 below: - 
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b+ 



Pa J 



with c w set to -Jk/200 . In practice the value 6=0.0803 has giv en good values. Thus 
the segmentation process is based on the value of*, calculated from the gradient map 
and so adapted to the particular image being processed. Areas where C > C MM are 
marked as microcalcifications. 

Figure 1 1 illustrates various original SMF images in Figures 1 1 A, C, E, G and I, 
with corresponding detection maps in Figures 1 IB, D, F, H and J. The marked areas are 
those areas where the contrast C is greater than minimum contrast C Wlr It can be seen, 
therefore, that the microcalcincauon are clearly visible in the segmented map 1 3 
illustrated in Figures 1 IB, D, F, H and J. 

Although the invention has been described in relation to the processing of 
mammograms and in particular the processing of mammograms in the Standard 
Mammogram Format, it should be appreciated that the techniques are applicable to 
mammograms which are not in that format, and also to other images, medical or non- 
medical. 

The invention also extends to a computer program for executing the image 
processing method on a suitably programmed computer system, to a computer readable 
storage medium carrying the computer program, and to an image processing apparatus 
for executing the image processing method. 



